Tag
9 articles
Learn how FlashMap, a new speculative decoding technique, achieves up to 6.14x faster LLM decoding by generating multiple tokens in parallel using flow maps.
Learn how speculative decoding helps vision-language AI models work faster by predicting what comes next, without sacrificing accuracy.
Jina AI has released jina-ocr-v1, a 3.4B parameter document parser with built-in speculative decoding that enables efficient OCR on low-budget GPUs. The model achieves high accuracy and speed, with open access via Hugging Face and hosted solutions through Jina Reader.
Liquid AI introduces LFM2.5-DSpark draft models that accelerate decoding by up to 3.18x without altering model outputs, using speculative decoding techniques.
Tencent open-sources AngelSpec, a torch-native framework for speculative decoding that introduces DFly, a block-diffusion drafter, and achieves significant speedups in LLM inference.
Learn how DSpark, a new AI framework from DeepSeek, speeds up text generation in AI models by making smart guesses and verifying only when necessary, without sacrificing accuracy.
Researchers at UC San Diego introduce DFlash, a new speculative decoding technique that drafts whole token blocks in parallel, achieving up to 15x throughput improvement on NVIDIA Blackwell.
EAGLE 3.1, developed by the EAGLE team, vLLM, and TorchSpec, tackles attention drift in LLM inference, enhancing speculative decoding stability for production use.
Learn how speculative decoding helps AI systems generate text faster without losing accuracy, using a fast guess-and-check method.